Display system, display method and image capture device

ABSTRACT

A display system includes a camera, a processor and a display. The camera is configured to shoot a first image and a second image in order. The processor is configured to generate a third image when a difference between the first image and the second image is larger than or equal to a preset difference value. The display is configured to display the first image and the third image in order when the difference is larger than or equal to the preset difference value. A display method and an image capture device are also disclosed herein.

CROSS - REFERENCE TO RELATED APPLICATION

This application claims priority to Taiwan Application Serial Number110109817, filed Mar. 18, 2021, which is herein incorporated byreference in its entirety.

BACKGROUND Technical Field

The present disclosure relates to a display technique. Moreparticularly, the present disclosure relates to a display system, adisplay method and an image capture device.

Description of Related Art

In traditional display method, when zooming operations are performed toa digital image shot by a camera, the zooming operations may suffer fromlimits of resolution. When the image is zooming in to the limits of theresolution, the image becomes blurred. In some approaches, artificialintelligence is introduced to a super resolution model, such that theimage is clarified. However, continuously performing operations by thesuper resolution model increases efficacy consumption of a system, andtime consumed is longer. Thus, techniques associated with thedevelopment for overcoming the problems described above are importantissues in the field.

SUMMARY

The present disclosure provides a display system. The display systemincludes a camera, a processor and a display. The camera is configuredto shoot a first image and a second image in order. The processor isconfigured to generate a third image when a difference between the firstimage and the second image is larger than or equal to a presetdifference value. The display is configured to display the first imageand the third image in order when the difference is larger than or equalto the preset difference value. The third image is a high resolutionversion of the second image.

The present disclosure provides a display method. The display methodincludes: generating a first image and a second image in order;generating a ratio associated with a difference between the first imageand the second image; when the ratio is larger than a preset ratio,displaying the first image; and when the ratio is smaller than thepreset ratio, generating a high resolution version of the second imageaccording to the second image, and displaying the high resolutionversion.

The present disclosure provides an image capture device. The imagecapture device includes a camera and a processor. The camera isconfigured to shoot a first image and a second image in order. Theprocessor is configured to generate a third image when a differencebetween the first image and the second image is larger than or equal toa preset difference value. When the difference is larger than or equalto the preset difference value, the processor is further is configuredto output the first image and the second image in order, and the thirdimage is a high resolution version of the second image.

It is to be understood that both the foregoing general description andthe following detailed description are by examples, and are intended toprovide further explanation of the disclosure as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present disclosure are best understood from the followingdetailed description when read with the accompanying figures. It isnoted that, in accordance with the standard practice in the industry,various features are not drawn to scale. In fact, the dimensions of thevarious features may be arbitrarily increased or reduced for clarity ofdiscussion.

FIG. 1 is a schematic diagram of a display system illustrated accordingto some embodiments of this disclosure.

FIG. 2 is a flowchart diagram of a display method illustrated accordingto some embodiments of this disclosure.

FIG. 3 is a schematic diagram of an image shot by a camera illustratedaccording to some embodiments of this disclosure.

FIG. 4 is a flowchart diagram of a display method illustrated accordingto some embodiments of this disclosure.

DETAILED DESCRIPTION

The following disclosure provides many different embodiments, orexamples, for implementing different features of the provided subjectmatter. Specific examples of components and arrangements are describedbelow to simplify the present disclosure. These are, of course, merelyexamples and are not intended to be limiting. For example, the formationof a first feature over or on a second feature in the description thatfollows may include embodiments in which the first and second featuresare formed in direct contact, and may also include embodiments in whichadditional features may be formed between the first and second features,such that the first and second features may not be in direct contact. Inaddition, the present disclosure may repeat reference numerals and/orletters in the various examples. This repetition is for the purpose ofsimplicity and clarity and does not in itself dictate a relationshipbetween the various embodiments and/or configurations discussed.

Further, spatially relative terms, such as “beneath,” “below,” “lower,”“above,” “upper”, “left,” “right” and the like, may be used herein forease of description to describe one element or feature's relationship toanother element(s) or feature(s) as illustrated in the figures. Thespatially relative terms are intended to encompass differentorientations of the device in use or operation in addition to theorientation depicted in the figures. The apparatus may be otherwiseoriented (rotated 90 degrees or at other orientations) and the spatiallyrelative descriptors used herein may likewise be interpretedaccordingly.

The terms applied throughout the following descriptions and claimsgenerally have their ordinary meanings clearly established in the art orin the specific context where each term is used. Those of ordinary skillin the art will appreciate that a component or process may be referredto by different names. Numerous different embodiments detailed in thisspecification are illustrative only, and in no way limits the scope andspirit of the disclosure or of any exemplified term.

It is worth noting that the terms such as “first” and “second” usedherein to describe various elements or processes aim to distinguish oneelement or process from another. However, the elements, processes andthe sequences thereof should not be limited by these terms. For example,a first element could be termed as a second element, and a secondelement could be similarly termed as a first element without departingfrom the scope of the present disclosure.

In the following discussion and in the claims, the terms “comprising,”“including,” “containing,” “having,” “involving,” and the like are to beunderstood to be open-ended, that is, to be construed as including butnot limited to. As used herein, instead of being mutually exclusive, theterm “and/or” includes any of the associated listed items and allcombinations of one or more of the associated listed items.

As used herein, “around”, “about” or “approximately” shall generallymean within 20 percent, preferably within 10 percent, and morepreferably within 5 percent of a given value or range. Numericalquantities given herein are approximate, meaning that the term “around”,“about” or “approximately” can be inferred if not expressly stated.

Reference will now be made in detail to the present embodiments of thedisclosure, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers are used in thedrawings and the description to refer to the same or like parts.

FIG. 1 is a schematic diagram of a display system 100 illustratedaccording to some embodiments of this disclosure. In some embodiments,the display system 100 is configured to display images, such as imagesM21-M23 shown in FIG. 1.

As illustratively shown in FIG. 1, the display system 100 includes acamera 110, a processor 120 and a display 130. In some embodiments, thedisplay system 100 is configured to shoot an object OB and display animage corresponding to the object OB. In the embodiment shown in FIG. 1,the camera 110 and the processor 120 are included in an image capturedevice 101.

In some embodiments, the camera 110 is configured to shoot the object OBto generate the corresponding images M11 and M12. In some embodiments,the camera 110 is configured to shoot the object OB continuously, togenerate the images M11 and M12 in order. In some embodiments, thecamera 110 outputs the images M11 and M12 in a streaming manner. In someembodiments, the camera 110 shoots the object OB and outputs the imagesM11 and M12 simultaneously.

In some embodiments, the camera 110 is implemented as a document camera.In some embodiments, the camera 110 is an USB video class device, butnot limited to this.

As illustratively shown in FIG. 1, the camera 110 includes a camera lens112, a support 114 and a control panel 116. In some embodiments, thecamera lens 112 is configured to shoot the object OB. The support 114 isconfigured to support the camera lens 112, such that the camera lens 112is aligned with the object OB. The control panel 116 is configured tocontrol at least one of the camera lens 112 or the support 114. Forexample, the control panel 116 controls the support 114 to move thecamera lens 112 to a position which is proper for shooting the objectOB. In various embodiments, the support 114 may be implemented as agooseneck type arm or a mechanical arm.

In some embodiments, the processor 120 is configured to process theimages M11 and M12 shot by the camera 110 to generate at least one ofthe images M21, M22 and M23. The images M21 and M22 correspond to theimages M11 and M12, respectively. In some embodiments, the image M23 isa high resolution version of the image M12.

In some embodiments, the processor 120 is coupled to the camera 110 andthe display 130. For example, the processor 120 is coupled to the camera110 and the display 130 through a universal serial bus (USB) to receiveat least one of the images M11 and M12, and transmits at least one ofthe images M21, M22 and M23 to the display 130.

In some embodiments, the processor 120 stores application programs 122,and operates according to the application programs 122. In someembodiments, the application programs 122 are at least one of softwareor firmware. In some embodiments, the processor 120 is configured toprocess at least one of the images M11 and M12 according to theapplication programs 122, such as zoom in, zoom out and/or crop. In someembodiments, the processor 120 is configured to generate the image M23according to the application programs 122 and the image M12. In someembodiments, the application programs 122 include super resolutionoperations using artificial intelligence. In some embodiments, theapplication programs 122 include super resolution operations usingeither one or combination of interpolation and neural network.

In some embodiments, the display 130 is configured to display at leastone of the images M21, M22 and M23.

FIG. 2 is a flowchart diagram of a display method 200 illustratedaccording to some embodiments of this disclosure. In some embodiments,the display method 200 is applied to the display system 100 shown inFIG. 1, but embodiments of present disclosure are not limited to this.In various embodiments, the display method 200 may be applied to otherkinds of display systems. For illustration purpose, operations of thedisplay method 200 are described below with components of the displaysystem 100 as an example.

As illustratively shown in FIG. 2, the display method 200 includesoperations S21-S28. In various embodiments, display methods provided bypresent disclosure include a part or combination of the operationsS21-S28.

At the operation S21, the camera 110 shot the object OB to generate theimage M11. The processor 120 transmits the image M21 corresponding tothe image M11 to the display 130. The display 130 displays the imageM21.

At the operation S22, the camera 110 shot the object OB to generate theimage M12.

At the operation S23, the processor 120 generates the image M22corresponding to the image M12 according to the image M12, anddetermines whether a resolution of the image M22 is smaller than orequal to a preset resolution. In some embodiments, the preset resolutionis 160×120, but embodiments of present disclosure are not limited tothis. In various embodiments, the preset resolution may be differentnumerical values.

If the processor 120 determines that the resolution of the image M22 issmaller than or equal to the preset resolution at the operation S23, thedisplay system performs the operation S24. At the operation S24, theprocessor determines whether a difference between the image M21 and M22is larger than or equal to a preset difference value. In variousembodiments, the processor 120 determines the difference between theimage M21 and M22 based on various parameters. For example, theprocessor 120 determines the difference according to at least one of apixel difference, a color difference or a feature value difference ofthe images M21 and M22. In some embodiments, the processor 120 processesthe image M21 as a signal, processes the image M22 as a noise,calculates a corresponding peak signal to noise ratio (PSNR), anddetermines the difference according to the PSNR.

If the processor 120 determines that the resolution of the image M22 islarger than the preset resolution at the operation S23, the displaysystem performs the operation S25. At the operation S25, the display 130is configured to display the image M22.

If the processor 120 determines that the difference between the imagesM21 and M22 is smaller than the preset difference value at the operationS24, the display system performs the operation S26. At the operationS26, the display 130 is configured to display the image M21.

If the processor 120 determines that the difference between the imagesM21 and M22 is larger than or equal to the preset difference value atthe operation S24, the display system performs the operation S27. At theoperation S27, the processor 120 is configured to generate the image M23according to the image M22. At the operation S28, the display 130 isconfigured to display the image M23.

FIG. 3 is a schematic diagram of an image shot by the camera 110illustrated according to some embodiments of this disclosure. FIG. 3includes images 301-303. Referring to FIG. 2 and FIG. 3, operations ofthe display method 200 are described below with the images 301-303.

In some embodiments, the image 301 corresponds to the images M11 andM21, and the images 302 and 303 correspond to the images M12, M22 andM23 under different circumstances. In some embodiments, a differencebetween the images 301 and 302 is smaller than the preset differencevalue, and a difference between the images 301 and 303 is larger thanthe preset difference value. In some embodiments, in response toamplitude of variation of the object OB being small, the camera shootsthe object OB and generates the images 301 and 302 in order according tothe variation of the object OB. In response to amplitude of variation ofthe object OB being large, the camera shoots the object OB and generatesthe images 301 and 303 in order according to the variation of the objectOB.

Three circumstances are described below as examples. In the threecircumstances described below, a resolution of the image 301 is largerthan the preset resolution.

In the first circumstance, resolutions of the images 302 and 303 arelarger than the preset resolution. Correspondingly, the display system100 performs the operation S25, and the display 130 displays the image302 or 303 directly according to the variation of the object OB. In thefirst circumstance, the processor 120 does not perform the superresolution operation.

In the second circumstance, the variation of the object OB correspondsto the image 302. Correspondingly, the display system 100 performs theoperation S26, and the display 130 displays the image 301. In the secondcircumstance, the processor 120 does not perform the super resolutionoperation.

In the third circumstance, the variation of the object OB corresponds tothe image 303. Correspondingly, the display system 100 performs theoperations S27-S28, the processor 120 performs the super resolutionoperation to the image 303 to generate a high resolution version of theimage 303, and the display 130 displays the high resolution version ofthe image 303. In which the high resolution version of the image 303 hasa resolution higher than the preset resolution.

In the three circumstances described above, resolutions of imagesdisplayed by the display 130 are higher than the preset resolution.

In some approaches, a processor performs high resolution operations toimages shot by a camera continuously, such that consumption of efficacyof a system is high, and operation time is long.

Compared to the above approaches, in some embodiments of presentdisclosure, the high resolution operations are not performed when theresolution of the image is larger than the preset resolution. The highresolution operations are also not performed, and the image 301 having ahigh resolution is displayed directly when the variation (such as thevariation corresponding to the images 301 and 302) of the object shot bythe camera is small. As a result, under the condition that a quality ofthe image displayed by the display 130 is maintained, the display system100 has a lower consumption of efficacy and shorter operation time.

FIG. 4 is a flowchart diagram of a display method 400 illustratedaccording to some embodiments of this disclosure. In some embodiments,the display method 400 is applied to the display system 100 shown inFIG. 1, but embodiments of present disclosure are not limited to this.In various embodiments, the display method 400 may be applied to otherkinds of display systems. For illustration purpose, operations of thedisplay method 400 are described below with components of the displaysystem 100 as an example.

As illustratively shown in FIG. 4, the display method 400 includesoperations S41, S499 and S414, in which the operation S499 includesoperations S42-S413. In various embodiments, display methods provided bypresent disclosure include a part or combination of the operationsS41-S414.

Referring to FIG. 4 and FIG. 2, the display method 400 is an alternativeembodiment of the display method 200. The operations S41, S47 S49 andS410 correspond to the operations S22, S23, S24 and S27, respectively,and the operation S414 corresponds to the operations S25, S26 and S28.Therefore, some descriptions are not repeated for brevity.

Before the operation S41, the display 130 is configured to display afirst image. At the operation S41, the camera 110 shoots the object OBto generate an original image of a second image, and transmit MJPEGformat data of the original image to the processor 120 through an USBline transmission.

At the operation S499, the processor 120 provides the first image, thesecond image and a third image to the display 130 by the applicationprogram 122 according to the second image.

At the operation S414, the display 130 displays the first image, thesecond image and the third image from the processor 120.

Further details of the operations S42-S413 in the operation S499 aredescribed below. At the operation S42, a receiver in the processor 120receives the original image shot by the camera 110.

At the operation S43, a decoder in the processor 120 decodes the MJPEGformat data of the original image to generate RGB image data of theoriginal image.

At the operation S44, a buffer in the processor 120 stores the RGB imagedata of the original image.

At the operation S45, a user performs zoom in/out and crop operations tothe original image to generate the second image corresponding to detailsthose the user desires to see. In some embodiments, the second image hasa resolution lower than a resolution of the original image.

At the operation S46, the buffer in the processor 120 stores the secondimage.

At the operation S47, the processor checks whether the resolution of thesecond image is smaller than or equal to the preset resolution. In theembodiment shown in FIG. 4, the preset resolution is 160×120.

If the processor 120 determines that the resolution of the second imageis larger than 160×120 at the operation S47, the processor 120 storesthe second image into the buffer configured to store an image fordisplaying corresponding to the operation S413. If the processor 120determines that the resolution of the second image is smaller than orequal to 160×120 at the operation S47, the operation S48 is performed.

At the operation S48, a PSNR module in the processor 120 generates acorresponding PSNR value according to the second image and the firstimage. In which the PSNR module processes the first image as a signaland processes the second image as a noise, to generate the PSNR value.In response to a difference between the second image and the first imagebeing smaller, the PSNR value is larger.

At the operation S49, the processor checks whether the PSNR value issmaller than or equal to a preset PSNR value. In the embodiment shown inFIG. 4, the preset PSNR value is 30 dB, but embodiments of presentdisclosure are not limited to this.

If the processor 120 determines that the PSNR value is larger than 30 dBat the operation S49, the processor 120 stores the first image into thebuffer corresponding to high resolution images corresponding to theoperation S412. If the processor 120 determines that the resolution ofthe second image is smaller than or equal to 30 dB at the operation S49,the operation S410 is performed.

At the operation S410, the processor 120 generates the third image byartificial intelligence according to the second image. The third imageis a high resolution version of the second image. In variousembodiments, the resolution of the third image may be twice, three timesor four times of the resolution of the second image, but embodiments ofpresent disclosure are not limited to this.

In some embodiments, the operation S410 includes the operation S411. Atthe operation S411, the processor 120 generates the third image by asuper resolution model of an efficient sub-pixel convolutional neuralnetwork (ESPCN) according to the second image.

At the operation S412, the buffer, in the processor 120, correspondingto high resolution images stores the first image or the third imageaccording to the operation S49.

At the operation S413, the buffer, in the processor 120, configured tostore an image for displaying stores the first image, the second imageor the third image according to the operations S412 and S47, and outputsthe stored images to the display 130.

At the operation S414, the display 130 displays the first image, thesecond image or the third image according to the image stored at theoperation S413.

In summary, in some embodiments of present disclosure, by performing theoperations S23, S24, S47 and S49, the display system 100 may not performthe super resolution operation when the resolution of the image ishigher than the preset resolution or the variation of the image issmall, such that the display system 100 has a lower consumption ofefficacy and a shorter operation time.

Although the present disclosure has been described in considerabledetail with reference to certain embodiments thereof, other embodimentsare possible. Therefore, the spirit and scope of the appended claimsshould not be limited to the description of the embodiments containedherein.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the structure of the presentdisclosure without departing from the scope or spirit of the disclosure.In view of the foregoing, it is intended that the present disclosurecover modifications and variations of this disclosure provided they fallwithin the scope of the following claims.

What is claimed is:
 1. A display system, comprising: a camera configured to shoot a first image and a second image in order; a processor configured to generate a third image when a difference between the first image and the second image is larger than or equal to a preset difference value; and a display configured to display the first image and the third image in order when the difference is larger than or equal to the preset difference value, wherein the third image is a high resolution version of the second image.
 2. The display system of claim 1, wherein the processor does not generate the third image when the difference is smaller than the preset difference value, and the display is further configured to display the first image when the difference is smaller than the preset difference value.
 3. The display system of claim 1, wherein the display is further configured to display the first image and the second image in order when a resolution of the second image is larger than a preset resolution.
 4. The display system of claim 3, wherein the processor does not generate the third image when the resolution of the second image is larger than the preset resolution.
 5. The display system of claim 1, wherein the processor is further configured to output the third image when a resolution of the second image is smaller than a preset resolution and the difference between the first image and the second image is larger than or equal to the preset difference value.
 6. The display system of claim 5, wherein the preset resolution is 160×120.
 7. The display system of claim 1, wherein the processor is further configured to calculate a peak signal to noise ratio by processing the first image as a signal and processing the second image as a noise, wherein the difference corresponds to the peak signal to noise ratio.
 8. The display system of claim 7, wherein the preset difference value corresponds to 30 dB.
 9. The display system of claim 1, wherein the third image is generated by an efficient sub-pixel convolutional neural network with artificial intelligence.
 10. The display system of claim 1, wherein the processor is further configured to store the first image or the third image when a resolution of the second image is smaller than or equal to 160×120.
 11. A display method, comprising: generating a first image and a second image in order; generating a ratio associated with a difference between the first image and the second image; when the ratio is larger than a preset ratio, displaying the first image; and when the ratio is smaller than the preset ratio, generating a high resolution version of the second image according to the second image, and displaying the high resolution version.
 12. The display method of claim 11, further comprising: when the ratio is smaller than the preset ratio, displaying the first image and the high resolution version of the second image in order.
 13. The display method of claim 12, further comprising: determining a resolution of the second image; and when the resolution is larger than or equal to a preset resolution, displaying the first image and the second image in order, without generating the high resolution version of the second image.
 14. The display method of claim 13, further comprising: when the resolution is smaller than the preset resolution and the ratio is larger than the preset ratio, displaying the first image, without generating the high resolution version of the second image.
 15. The display method of claim 14, further comprising: when the resolution is smaller than the preset resolution and the ratio is smaller than or equal to the preset ratio, displaying the first image and the high resolution version of the second image in order.
 16. The display method of claim 15, wherein the preset resolution is 160×120, and the preset ratio is 30 dB.
 17. An image capture device, comprising: a camera configured to shoot a first image and a second image in order; and a processor configured to generate a third image when a difference between the first image and the second image is larger than or equal to a preset difference value, wherein when the difference is larger than or equal to the preset difference value, the processor is further is configured to output the first image and the second image in order, and the third image is a high resolution version of the second image.
 18. The image capture device of claim 17, wherein the processor is further configured to output the third image when a resolution of the second image is smaller a preset resolution and the difference between the first image and the second image is larger than or equal to the preset difference value.
 19. The image capture device of claim 18, wherein the processor does not generate the third image when the difference is smaller than the preset difference value or when the resolution is larger than or equal to the preset resolution.
 20. The image capture device of claim 19, wherein the processor is further configured to output the first image and the second image in order when the resolution is larger than or equal to the preset resolution and the difference is larger than or equal to the preset difference value. 